Image processing device and image processing method

ABSTRACT

An image processing device includes a plurality of mixer units, a plurality of scale processing units, and an enable signal processing unit. A mixer unit of the plurality of the plurality of mixer units selects one of an output of another mixer unit of the plurality of mixer units, and an output image of a scale processing unit of the plurality of scale processing units to generate a composite image.

REFERENCE TO RELATED APPLICATION

This application is a Continuation application of U.S. patentapplication Ser. No. 13/463,727, which was filed on May 3, 2012, and thedisclosure of which is incorporated herein in its entirety by referencethereto.

CROSS-REFERENCE TO RELATED APPLICATIONS

The disclosure of Japanese Patent Application No. 2011-119144 filed onMay 27, 2011 including the specification, drawings, and abstract isincorporated herein by reference in its entirety.

BACKGROUND

The present invention relates to an image processing device and an imageprocessing method.

In recent years, it is highly necessary that personal computers, gamingdevices, digital-broadcast-ready television sets, Blu-ray recorders, andother digital home appliances have a three-dimensional (3D) capability.More specifically, it is necessary that these devices accept an input 3Dimage and properly display the 3D image.

The 3D image input format for the above-mentioned devices has beendiversified. The 3D image output format for the devices has also beendiversified. In consideration of the diversified 3D image input/outputformat, some models of television devices having a 3D image displaycapability have newly incorporated various 3D signal processingfunctions in addition to a 3D image display function compliant with anexisting digital broadcast standard. However, these models have entailedincreased cost, including the cost of development and manufacture andthe cost of overall system, because they have special functions such asthe 3D signal processing functions.

A method of generating a 3D stereoscopic image, which is to be inputinto a display functionality of the above devices, is disclosed in USPatent Publication No. 2010/0302235. The method disclosed in US PatentPublication No. 2010/0302235 uses a pixel blender (synthesis means),such as a GPU (Graphics Processing Unit), to generate a 3D stereoscopicimage from plural images (two images in most cases) and maskinformation.

FIG. 15 is a diagram illustrating a method of generating a 3Dstereoscopic image in a 3D television system described in US PatentPublication No. 2010/0302235. A method of generating a 3D stereoscopicimage 53 from a left eye image 51 and a right eye image 52 is describedbelow. The 3D stereoscopic image 53 is structured so that the pixelvalues of pixels included in the left eye image are set for pixels inodd-numbered lines while the pixel values of pixels included in theright eye image are set for pixels in even-numbered lines. The methoddescribed in US Patent Publication No. 2010/0302235 uses pixel blenders60, 61, which are operated by a CPU or GPU. The pixel blender 61 readsthe left eye image 51 and mask information 54 from a storage device suchas a RAM (Random Access Memory). The pixel blender 60 performs a processdefined by Porter-Duff's “A atop B” rule. More specifically, the pixelblender 60 sets the pixel value of a pixel in the corresponding positionof the left eye image 51 for a pixel for which the value “1” isdesignated in the mask information 54. The pixel blender 60 does not seta pixel value for a pixel for which the value “0” is designated in themask information 54. The pixel blender 60 generates an intermediateimage 55 by performing the above-described process for all pixels. Thepixel blender 60 writes the generated intermediate image 55 in thestorage device such as a RAM.

The pixel blender 61 reads the right eye image 52 and the intermediateimage 55 from the storage device such as a RAM. The pixel blender 61performs a process defined by Porter-Duff's “A over B” rule. Morespecifically, the pixel blender 61 sets the pixel value of a pixel inthe corresponding position of the right eye image 52 for a pixel forwhich the value “0” is designated in the intermediate image 55. Thepixel blender 61 does not perform the process for a pixel for which thepixel value derived from the left eye image is already set. The pixelblender 61 generates an output image 53 by performing theabove-described process for all pixels. The pixel blender 61 writes thegenerated output image 53 in the storage device such as a RAM.

FIG. 16 is a diagram illustrating the relationship between each processand memory access (access to the storage device) that occurs when themethod described in US Patent Publication No. 2010/0302235 is used. Thepixel blender 60 performs a process of reading the storage device threetimes. The pixel blender 61 performs a process of reading the storagedevice two times. The pixel blenders 60, 61 perform a process of writinginto the storage device once. Further, another processing unit performsa process of reading the output image 53 from the storage device once.

Plane synthesis, which is described in “Data Broadcast Encoding Methodand Transmission Method in Digital Broadcasting” (retrieved from theInternet on Apr. 10, 2011; URL:http//www.arib.or.jp/english/html/overview/doc/2-STD-B24v5_(—)1-1p3.pdf),is suggested in Japanese Unexamined Patent Publications No. 2003-324652and 2010-068399. A plane synthesis technology described in “DataBroadcast Encoding Method and Transmission Method in DigitalBroadcasting” will now be described with reference to FIG. 17. The planesynthesis technology defines the plane synthesis for digitalbroadcasting.

FIG. 17 is a schematic diagram illustrating the digital broadcast planesynthesis technology described in “Data Broadcast Encoding Method andTransmission Method in Digital Broadcasting”. The term “plane” refers toa display screen for displaying mono-media (independent expression mediasuch as video, audio, text, and still images).

A mixer unit 70 reads still image data 81 from a RAM 80 as video/stillplane 0 (91). The mixer unit 70 reads moving image data 82 from the RAM80 as video/still plane 1 (92). Video/still plane 0 (91) and video/stillplane 1 (92) may be either a moving image or a still image.

Similarly, the mixer unit 70 reads text data 84 from the RAM 80 as asubtitle plane 94. The mixer unit 70 reads text/graphic data 85 from theRAM 80 as a text/graphic plane 95. Digital broadcast requirementsprescribe that five planes be handled as shown in FIG. 17. However, alager number of planes may be handled. For example, a plane fordisplaying a user's operating menu and a plane for displaying a cursormay be additionally handled.

The mixer unit 70 reads a switching plane 83 from the RAM 80. As regardsthe switching plane 83, the setup for a transparentizing process isdefined for a situation where two planes (video/still plane 0 (91) andvideo/still plane 1 (92) in the current example) are synthesized. As forthe switching plane 83, 1-bit data is set for each pixel. The 1-bit datais either a value (“0”) indicating the setup of a pixel value forvideo/still plane 0 (91) or a value (“1”) indicating the setup of apixel value for video/still plane 1 (92).

The mixer unit 70 can independently set a scaling ratio for video/stillplane 0 (91) and video/still plane 1 (92). In other words, the mixerunit 70 can individually enlarge or reduce video/still plane 0 (91) andvideo/still plane 1 (92) while keeping them independent of each other.

The mixer unit 70 reduces video/still plane 9 (91) in accordance with aregion defined by the switching plane 83 (video/still plane 9 (91) inFIG. 17 is reduced to ½). Further, the mixer unit 70 examines a bitvalue that is set for each pixel of the switching plane 83. If the value“0” is set as the bit value for a particular pixel, the mixer unit 70operates so that the pixel value of a target pixel of video/still plane0 (91) is set for the particular pixel. If, on the other hand, the value“1” is set as the bit value for a particular pixel, the mixer unit 70operates so that the pixel value of a target pixel of video/still plane1 (92) is set for the particular pixel. In this manner, the mixer unit70 generates an intermediate image (not shown) that is a combination ofvideo/still plane 0 (91) and video/still plane 1 (92).

The mixer unit 70 superimposes the subtitle plane 94 and thetext/graphic plane 95 over the intermediate image to generate an outputimage 96.

Although “Data Broadcast Encoding Method and Transmission Method inDigital Broadcasting” suggests and teaches the digital broadcast planesynthesis technology, it does not describe the generation of astereoscopic image at all.

SUMMARY

As indicated in FIG. 16, the method described in US Patent PublicationNo. 2010/0302235 needs to read a storage device multiple times in orderto generate the intermediate image 55 and the output image 53. Thismethod also needs to write the intermediate image 55 and the outputimage 53 into the storage device. This not only increases the amount ofsystem resources required to acquire the output image 53, such as therequired memory capacity, memory bandwidth, and processing time, butalso increases overall power consumption. This problem is evidentparticularly when the method described in US Patent Publication No.2010/0302235 is used to generate a stereoscopic image from ahigh-resolution moving image.

According to one aspect of the present invention, there is provided animage processing device including a mixer unit. The mixer unit mixes afirst image with a second image to generate an output image inaccordance with first mask information, which defines a display regionso that substantially the same number of pixels in the first and secondimages are displayed while the pixels are dispersed substantiallyuniformly.

According to another aspect of the present invention, there is providedan image processing method including the step of mixing a first imagewith a second image to generate a stereoscopic output image inaccordance with first mask information, which defines a display regionso that substantially the same number of pixels in the first and secondimages are displayed while the pixels are dispersed substantiallyuniformly.

According to an aspect of the present invention, the mask informationprescribes that substantially the same number of pixels in the first andsecond images be uniformly disposed for stereoscopic viewing while thefirst image corresponds to one eye of a user and the second imagecorresponds to the other eye. The mixer unit uses the mask informationto mix the first image with the second image for the purpose ofgenerating a stereoscopic image. In this instance, the mixer unit cangenerate the stereoscopic image without creating an intermediate image.

The present invention makes it possible to provide an image processingdevice and image processing method that are capable of generating astereoscopic image with limited system resources.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will be described in detail basedon the following figures, in which:

FIG. 1 is a block diagram illustrating the configuration of an imageprocessing device according to a first embodiment of the presentinvention;

FIG. 2 is a block diagram illustrating the details of a mixer unitaccording to the first embodiment;

FIG. 3 is another block diagram illustrating the details of the mixerunit according to the first embodiment;

FIG. 4 is another block diagram illustrating the configuration of theimage processing device according to the first embodiment;

FIG. 5 is still another block diagram illustrating the configuration ofthe image processing device according to the first embodiment;

FIG. 6 is a diagram illustrating mask information according to the firstembodiment;

FIG. 7 is a diagram illustrating the relationship between an operationof the mixer unit according to the first embodiment and memory access(access to a storage device);

FIG. 8 is a block diagram illustrating the configuration of the imageprocessing device according to a second embodiment of the presentinvention;

FIG. 9 is a block diagram illustrating the configuration of a scaleprocessing unit according to the second embodiment;

FIG. 10 is a block diagram illustrating the configuration of an enablesignal processing unit according to the second embodiment;

FIG. 11 is a diagram illustrating the relationship between an input intoand an output from the enable signal processing unit according to thesecond embodiment;

FIG. 12 is a diagram illustrating a stereoscopic image generationoperation performed by the image processing device according to thesecond embodiment;

FIG. 13 is a diagram illustrating a stereoscopic image data generationoperation that is performed by the image processing device according tothe second embodiment when side-by-side image data is input;

FIG. 14 is a diagram illustrating a side-by-side image data generationoperation performed by the image processing device according to thesecond embodiment;

FIG. 15 is a diagram illustrating a method of generating a 3Dstereoscopic image in a 3D television system described in US PatentPublication No. 2010/0302235;

FIG. 16 is a diagram illustrating the relationship between each processand memory access (access to a storage device) that occurs when a methoddescribed in US Patent Publication No. 2010/0302235 is used; and

FIG. 17 is a diagram illustrating a plane synthesis technology describedin “Data Broadcast Encoding Method and Transmission Method in DigitalBroadcasting” (retrieved from the Internet on Apr. 10, 2011; URL:http//www.arib.or.jp/english/html/overview/doc/2-STD-B24v5_(—)1-1p3.pdf).

DETAILED DESCRIPTION First Embodiment

A first embodiment of the present invention will now be described withreference to the accompanying drawings. FIG. 1 is a block diagramillustrating the configuration of an image processing device 1 accordingto the first embodiment. The image processing device 1 may be, forexample, a television set, a set-top box, a digital signage device, amobile phone having telephoning and gaming functions, and a projector.

The image processing device 1 includes a mixer unit 10 and a storagedevice (not shown). The storage device is a so-called memory such as aRAM (Random Access Memory). The storage device need not always bemounted in the image processing device 1. For example, it may be a flashmemory or other similar external device.

The mixer unit 10 accesses the storage device (not shown) and readsfirst image data 21, which is a moving image or a still image, andsecond image data 22, which is a moving image or a still image. Thefirst image data 21 is an image corresponding to a user's right eye in astereoscopic image. The second image data 22 is an image correspondingto a user's left eye in the stereoscopic image. The mixer unit 10 alsoreads mask information 23 from the storage device. The mask information23 defines a display region of the first image data 21 and of the secondimage data 22.

The mask information 23 defines a range within which the pixel value ofeach pixel included in the first image data 21 is used to generate astereoscopic image and a range within which the pixel value of eachpixel included in the second image data 22 is used to generate thestereoscopic image. The mask information 23 has the same number ofpixels as the first image data 21 and the second image data 22.

For each pixel included in the mask information 23, either the bit value“0” or the bit value “1” is set. The bit value “0” indicates that thefirst image data 21 is to be displayed. The bit value “1” indicates thatthe second image data 22 is to be displayed. In the mask information 23,the number of pixels for which the bit value “0” is set issubstantially, or preferably exactly, equal to the number of pixels forwhich the bit value “1” is set.

When the pixels for which the bit value “0” is set and the pixels forwhich the bit value “1” is set are disposed in the mask information 23,they are dispersed substantially uniformly. In the mask information 23shown, for instance, in FIG. 1, the pixels for which the bit value “0”is set and the pixels for which the bit value “1” is set are alternatelydisposed in each column. In other words, the mask information 23 shownin FIG. 1 is configured so that the bit values are disposed in ahorizontal stripe pattern (line alternate).

The mixer unit 10 is a processing unit that generates output image data24 by using the first image data 21, the second image data 22, and themask information 23. In the present embodiment, the output image data 24is stereoscopic image data. The mixer unit 10 reads the first image data21, the second image data 21, and the mask information 23 from thestorage device and outputs the output image data 24.

FIG. 2 is a block diagram illustrating the details of the mixer unit 10.The mixer unit 10 includes a selection signal output unit 11 and aswitch unit 12.

The selection signal output unit 11 reads a bit value from the maskinformation 23 on an individual pixel basis. When the read bit value is“0”, the selection signal output unit 11 outputs a signal for selectingthe pixel value of the first image data 21 (image corresponding to theright eye) at relevant coordinates to the switch unit 12. When the readbit value is “1”, the selection signal output unit 11 outputs a signalfor selecting the pixel value of the second image data 22 (imagecorresponding to the left eye) at relevant coordinates to the switchunit 12. More specifically, a selection signal output from the selectionsignal output unit 11 includes information about coordinates andinformation for specifying the pixel value to be set at the coordinates.

The switch unit 12 reads a pixel value from relevant coordinates of thefirst image data 21 or the second image data 22 in accordance with theselection signal supplied from the selection signal output unit 11.Further, the switch unit 12 sets the read pixel value at the relevantcoordinates of the output image data 24.

The selection signal output unit 11 and the switch unit 12 perform theabove-described processes for all pixels included in the maskinformation 23. This causes the mixer unit 10 to generate a stereoscopicimage 24.

The mixer unit 10 may alternatively be configured as shown in FIG. 3.FIG. 3 is a block diagram illustrating an alternative configuration ofthe mixer unit 10. The mixer unit 10 includes an alpha blend valuecalculation unit 13, a multiplication unit 14, a subtraction unit 15, amultiplication unit 16, and an addition unit 17.

The alpha blend value calculation unit 13 reads a bit value from themask information 23 on an individual pixel basis. When the read bitvalue is “0”, the alpha blend value calculation unit 13 sets an alphablend value at relevant coordinates to “0”. When the read bit value is“1”, the alpha blend value calculation unit 13 sets the alpha blendvalue at the relevant coordinates to “1”. The alpha blend valuecalculation unit 13 notifies the multiplication unit 14 and subtractionunit 15 of the set alpha blend value and the information about thecoordinates.

The multiplication unit 14 reads the pixel value at the relevantcoordinates from the second image data 22. The multiplication unit 14multiplies the read pixel value by the alpha blend value. Themultiplication unit 14 supplies the result of multiplication to theaddition unit 17.

The subtraction unit 15 subtracts the supplied alpha blend value from 1and supplies the result of subtraction (1−a) to the multiplication unit16. The multiplication unit 16 reads the pixel value of the relevantcoordinates from the first image data 21. The multiplication unit 16multiplies the read pixel value by the alpha blend value that issupplied from the subtraction unit 15 as the result of subtraction. Themultiplication unit 16 supplies the result of multiplication to theaddition unit 17.

The addition unit 17 adds the pixel value supplied from themultiplication unit 14 to the pixel value supplied from themultiplication unit 16, and sets the result of addition as the pixelvalue at the relevant coordinates.

Each processing unit (alpha blend value calculation unit 13,multiplication unit 14, subtraction unit 15, multiplication unit 16, andaddition unit 17) included in the mixer unit 10 sets the above-describedpixel value for all pixels included in the mask information 23. Thiscauses the mixer unit 10 to generate a stereoscopic image 24.

The array of bits in the mask information 23 (the array of pixels forwhich the bit value “0” is set and of pixels for which the bit value “1”is set) is not limited to the horizontal stripe pattern shown in FIG. 1.Any bit array is acceptable as far as it is prescribed thatsubstantially the same number of pixels in the first and second imagedata 21, 22 be dispersed substantially uniformly when they aredisplayed.

FIGS. 4 and 5 show other examples of the mask information 23. Theexample shown in FIG. 4 indicates that the bit array in the maskinformation 23 is a vertical stripe pattern (dot by dot). The exampleshown in FIG. 5 indicates that the bit array in the mask information 23is a checkered pattern (checkerboard).

The mixer unit 10 may use two different sets of mask information 23alternately at fixed intervals. FIG. 6 shows two different sets of maskinformation 23. Two sets of mask information 23, which differ in the bitvalue position by one bit only, are prepared as shown in the figure. Themixer unit 10 uses the two sets of mask information 23 alternately atfixed intervals. For example, the mixer unit 10 may internally cachethese sets of mask information 23 and switch between the cached sets ofmask information 23 at fixed intervals. This ensures that the left-eyepixel position and right-eye pixel position change at fixed intervalswithout being definitely fixed. As the left-eye pixel position andright-eye pixel position change at fixed intervals, a stereoscopic imagecan be properly displayed even when, for instance, the image has avertical stripe pattern.

The relationship between an operation of the mixer unit 10 in the imageprocessing device 1 and memory access (access to the storage device)will now be described. FIG. 7 is a diagram illustrating the relationshipbetween the operation of the mixer unit 10 and memory access (access tothe storage device).

As shown in the figure, the mixer unit 10 reads the first image data 21,the second image data 22, and the mask information 23 from the storagedevice. The mixer unit 10 then generates the output image data 24 in theearlier described manner from the read first image data 21, second imagedata 22, and mask information 23. The mixer unit 10 can directly supplythe generated output image data 24 to any processing unit.

As described above, the mixer unit 10 reads data from the storage devicethree times to acquire display data. In this instance, the mixer unit 10does not write data into the storage device.

Advantages provided by the image processing device 1 according to thepresent embodiment will now be described. As described above, the imageprocessing device 1 according to the present embodiment generates thestereoscopic image 24 without generating an intermediate image. Thiseliminates the necessity of using an extra memory space for storing theintermediate image and acquiring a memory bandwidth for the intermediateimage.

Further, as the image processing device generates the stereoscopic image24 without generating the intermediate image, it can reduce the timerequired for processing. Furthermore, as the time required forprocessing is reduced, the image processing device 1 according to thepresent embodiment can reduce power consumption as compared to themethod described in US Patent Publication No. 2010/0302235.

Second Embodiment

A second embodiment of the present invention will now be described. Theimage processing device according to the second embodiment not onlyperforms a process of generating a stereoscopic image but also achievesa plane synthesis for digital broadcasting. The image processing deviceaccording to the present embodiment is described below.

FIG. 8 is a block diagram illustrating the configuration of the imageprocessing device according to the present embodiment. The imageprocessing device 1 includes scale processing units 30-33, mixer units101-103, and an enable signal processing unit 40.

FIG. 8 shows an example in which the image processing device 1 shown inthe figure achieves a plane synthesis for digital broadcasting. Thefirst image data 21 and the second image 22 are either a still image ora moving image. The mask information 23 corresponds to a so-calledswitching plane. Text/image data 25 corresponds to a text/graphic planeand a subtitle plane. The output image data 24, which is output from theimage processing device 1, is a composite image that is obtained byplacing the first and second image data 21, 22 in the background andplacing the text/image data 25 in the foreground. An operation of eachprocessing unit and a method of generating the output image data 24 aredescribed below.

The scale processing unit 30 cuts out a necessary rectangle from thefirst image data 21, enlarges or reduces the rectangle to apredetermined size, and supplies the enlarged or reduced rectangularportion of the first image data 21 to the mixer unit 101. The scaleprocessing unit 30 generates an enable signal for pixel value selectionand supplies the enable signal to the mixer unit 101. The details of thescale processing unit 30 are described below with reference to FIG. 9.

The scale processing unit 30 includes a cutting unit 301 and anenlarging/reducing unit 302. The cutting unit 301 reads data selectioninformation 303 from the storage device. The data selection information303 indicates what pixels are to be cut out from the first image data21. The data selection information 303 and the first image data 21 areequal in image size (in the number of vertical and horizontal pixels).Either the bit value “0” or the bit value “1” is set for each pixel inthe data selection information 303. Pixels for which the bit value “1”is set are to be cut out. In other words, the data selection information303 defines the display target region of the first image data 21.

The cutting unit 301 cuts out the region defined by the data selectioninformation 303 from the first image data 21. The cutting unit 301 thensupplies the cut-out image 304 to the enlarging/reducing unit 302.

The enlarging/reducing unit 302 reads display position information 305from the storage device. The display position information 305 indicatesa display region (display position and display size) within an outputimage 306 generated from the scale processing unit 30 in which thecut-out image 304 is to be displayed. In other words, the displayposition information 305 defines the scale of the cut-out image 304. Theenlarging/reducing unit 302 enlarges or reduces the cut-out image 304 inaccordance with the display position information 305 to generate theoutput image 306. In short, the output image 306 is processed firstimage data 21. Further, the enlarging/reducing unit 302 generates anenable signal 307 that indicates the effective region of the outputimage 306. The enable signal 307 has the same bit pattern as the displayposition information 305. The bit value (either “0” or “1”) of eachpixel included in the enable signal 307 is used for later-describedpixel value selection in the mixer units 101-103. A 1-bit signal thatcorresponds to each pixel of the output image 306 and is used for pixelvalue selection as described above is referred to as the enable signal

In reality, the cut-out image 304, the output image 306, and the enablesignal 307 are exchanged as signals between the individual processingunits. In other words, the cut-out image 304, the output image 306, andthe enable signal 307 need not be written into the storage device.

Referring again to FIG. 8, the scale processing unit 31 uses dataselection information 313 and display position information 315. Theinternal configuration of the scale processing unit 31 is the same asshown in FIG. 9. The scale processing unit 32 and the scale processingunit 33 are also configured as shown in FIG. 9. Display positioninformation 325 concerning the mask information 23 can be interpreted asan effective region within which the mask information is effective.

The enable signal processing unit 40 generates an enable signal that isto be supplied to the mixer unit 102. The configuration of the enablesignal processing unit 40 is described below with reference to FIG. 10.

The enable signal processing unit 40 includes a NAND gate 401 and an ANDgate 402. An output image 326 and enable signal 327 are input into theNAND gate 401. As the mask information 23 corresponds to a switchingplane, a bit value is set for each pixel. Therefore, each pixel of theoutput image 326 has either the bit value “0” or the bit value “1”. Inother words, either the bit value “0” or the bit value “1” is one inputof the NAND gate 401.

The output of the NAND gate 401 and enable signal 317 are input into theAND gate 402. The AND gate 40 supplies a logical sum of these two inputsto the mixer unit 102.

FIG. 11 is a diagram illustrating the relationship between an input intoand an output from the enable signal processing unit 40. As shown in thefigure, when the enable signal 327 is “0”, the bit value of the enablesignal 317 is output as an enable signal without regard to the bit valueof the output image 326.

When the enable signal 327 is “1”, the bit value of the output image 326is “1”. The bit value output as the enable signal is “1” only when theenable signal 327 is “1”.

Referring back to FIG. 8, the mixer units 101-103 have the sameconfiguration as the mixer unit shown in FIG. 2. The mixer unit 101 usesbackground information 26. The background information 26 is image datafor which a predetermined plain color is designated, and need not bestored in the storage device.

When the bit value of the enable signal 307 is “0”, the mixer unit 101outputs the pixel value at relevant coordinates of the backgroundinformation 26 as the pixel value at the relevant coordinates. When, onthe other hand, the bit value of the enable signal 307 is “1”, the mixerunit 101 outputs the pixel value at relevant coordinates of the outputimage 326, which is output from the scale processing unit 30, as thepixel value at the relevant coordinates.

In accordance with the enable signal output from the enable signalprocessing unit 40, the mixer unit 102 selects and outputs either apixel value output from the mixer unit 101 or a pixel value output fromthe scale processing unit 31. When the enable signal output from theenable signal processing unit 40 is “0”, the mixer unit 102 selects thepixel value output from the mixer unit 101 and outputs it to the mixerunit 103. When, on the other hand, the enable signal output from theenable signal processing unit 40 is “1”, the mixer unit 102 selects thepixel value output from the scale processing unit 31 and outputs it tothe mixer unit 103.

When the bit value of an enable signal 337 is “0”, the mixer unit 103selects a pixel value output from the mixer unit 102 and sets it as thepixel value of a relevant pixel of the output image data 24. When, onthe other hand, the bit value of the enable signal 337 is “1”, the mixerunit 103 selects a pixel value output from the scale processing unit 33and sets it as the pixel value of the relevant pixel of the output imagedata 24.

The above description is given on the assumption that the mixer units101-103 are configured as shown in FIG. 2. However, the presentinvention is not limited to such a mixer unit configuration. Forexample, the mixer units 101-103 may be configured to perform an alphablend process as shown in FIG. 3.

A method of generating stereoscopic image data when the image processingdevice 1 is configured as shown in FIG. 8 will now be described withreference to FIG. 12. In FIG. 12, the output image data 24 isstereoscopic image data. The image processing device shown in FIG. 12has the same configuration as the image processing device 1 shown inFIG. 8. Processing units having the same name and reference numerals asthose described earlier perform the earlier-described operations. Whenstereoscopic image data is to be generated, the first image data 21 isan image corresponding to the user's right eye whereas the second imagedata 22 is an image corresponding to the user's left eye.

When the stereoscopic image data is to be generated, the data selectioninformation 303 designates the entire region of the first image data 21.The display position information 305 designates the entire region of theoutput image data 24. Similarly, the data selection information 313designates the entire region of the second image data 22. The displayposition information 315 designates the entire region of the outputimage data 24. The mask information 23 has a bit array in a verticalstripe pattern as shown, for instance, in FIG. 1. The bit array of themask information 23 is acceptable as far as substantially the samenumber of bit values are dispersed substantially uniformly. It may be ina horizontal stripe pattern or in a checkered pattern. Data selectioninformation 323 designates the entire region of the mask information 23.The display position information 325 designates the entire region of theoutput image data 24. Display position information 335 is set so as notto select an image region (namely, the bit value “0” is set for allpixels).

Upon completion of the above input, the mixer unit 101 outputs the firstimage data 21 that is rescaled to the same size as the output image data24. The mixer unit 102 outputs the second image data 22 that is rescaledto the same size as the output image data 24.

The enable signal processing unit 40 outputs an enable signal as shownin FIG. 11. When the display position information 305 and the displayposition information 315 are set as described above, the enable signals317, 327 are always “1”. Therefore, for a pixel for which the bit value“1” is set in the mask information 23, the enable signal processing unit40 outputs the bit value “1” as an enable signal. On the other hand, fora pixel for which the bit value “0” is set in the mask information 23,the enable signal processing unit 40 outputs the bit value “0” as anenable signal.

Consequently, the mixer unit 102 generates stereoscopic image data. Asthe display position information 305 is set so as not to select an imageregion, the mixer unit 103 directly sets a signal input from the mixerunit 102 as the pixel value for the output image data 24.

In the above description, it is assumed that the scale processing units30-33 include an enlarging/reducing unit. However, theenlarging/reducing unit may be omitted if the input data (first imagedata 21, second image data 22, mask information 23, etc.) has the sameimage size as the output image data 24.

The above-mentioned mixer unit 102 mixes the rescaled first image data21 with the rescaled second image data 22 by using an enable signalgenerated in accordance with the mask information 23. The mixer unit 102generates stereoscopic image data by performing the above mixingoperation. It means that the mixer unit 102 has substantially the samerole as the mixer unit 10 shown in FIG. 2.

A method of generating stereoscopic image data by inputting side-by-sideimage data when the image processing device 1 is configured as shown inFIG. 8 will now be described with reference to FIG. 13.

The first image data 21 is structured so that the left half of itsregion is an image corresponding to the user's right eye, and that theright half of its region is an image corresponding to the user's lefteye. The data selection information 303 designates the right half regionof the first image data 21 (sets the bit value “1” for each pixel in theright half region). The display position information 305 designates theentire region of the output image data 24. The data selectioninformation 313 designates the left half region of the first image data21 (sets the bit value “1” for each pixel in the left half region). Thedisplay position information 315 designates the entire region of theoutput image data 24. The mask information 23 has a bit array in ahorizontal stripe pattern as shown, for instance, in FIG. 1. The dataselection information 323 designates the entire region of the maskinformation 23. The display position information 325 designates theentire region of the output image data 24. The display positioninformation 335 is set so as not to select an image region (namely, thebit value “0” is set for all pixels).

In accordance with the above input, the scale processing unit 30 outputsan image that is obtained by enlarging the right half region of thefirst image data to the data size of the output image data 24 (outputs,actually, the pixel value of each pixel included in the image). Themixer unit 101 outputs the image output from the scale processing unit30 on an “as is” basis.

In accordance with the above input, the scale processing unit 31 outputsan image that is obtained by enlarging the left half region of the firstimage data to the data size of the output image data 24 (outputs,actually, the pixel value of each pixel included in the image).

As is the case shown in FIG. 12, the enable signal processing unit 40outputs the bit value “1” as an enable signal for a pixel for which thebit value “1” is set in the mask information 23. For a pixel for whichthe bit value “0” is set in the mask information 23, on the other hand,the enable signal processing unit 40 outputs the bit value “0” as anenable signal.

The operations of the mixer units 102, 103 are the same as indicated inFIG. 12 and will not be described in detail.

A method of generating side-by-side image data when the image processingdevice 1 is configured as shown in FIG. 8 will now be described withreference to FIG. 14.

The first image data 21 is an image corresponding to the user's righteye. The data selection information 303 designates the entire region ofthe first image data 21 (sets the bit value “1” for all pixels). Thedisplay position information 305 designates the right half region of theoutput image data 24. The second image data 22 is an image correspondingto the user's left eye. The data selection information 313 designatesthe entire region of the second image data 22 (sets the bit value “1”for all pixels). The display position information 315 designates theleft half region of the output image data 24. The display positioninformation 325 and the display position information 335 are set so asnot to select an image region (namely, the bit value “0” is set for allpixels). As regards the mask information 23, the bit value “0” is setfor all pixels.

In accordance with the above input, the scale processing unit 30rescales the first image data 21 to the right half of the data size ofthe output image data 24, and outputs the rescaled image to the mixerunit 101. The mixer unit 101 outputs the image whose right half is thefirst image data 21 to the mixer unit 101.

In accordance with the above input, the scale processing unit 31rescales the second image data 22 to the left half of the data size ofthe output image data 24, and outputs the rescaled image to the mixerunit 102. Further, the scale processing unit 31 outputs the enablesignal 317 to the enable signal processing unit 40 in accordance withthe display position information 315.

As the bit value “0” is set for all pixels in the display positioninformation 325, the scale processing unit 32 always outputs the bitvalue “0” to the enable signal processing unit 40 as the enable signal327. Further, the scale processing unit 32 outputs the bit value “0” forall pixels to the enable signal processing unit 40 as the output image326.

In accordance with the above input, the enable signal processing unit 40generates an enable signal that provides the bit value “1” for pixelscorresponding to the left half region of the output image data 24 andthe bit value “0” for pixels corresponding to the right half region ofthe output image data 24, and outputs the generated enable signal to themixer unit 102.

In accordance with the enable signal, the mixer unit 102 generates animage by setting the pixel value of the second image data 22 for eachpixel in the left half region of the output image data 24 and settingthe pixel value of the first image data 21 for each pixel in the righthalf region of the output image data 24, and outputs the generatedimaged to the mixer unit 103.

As the bit value “0” is set for all pixels in the display positioninformation 315, the scale processing unit 33 always outputs the bitvalue “0” as the enable signal 337.

As the bit value “0” is always supplied as the enable signal 337, themixer unit 103 outputs the output of the mixer unit 102 on an “as is”basis. The image processing device 1 generates the output image data 24,which is a side-by-side image, by performing the above-described seriesof processes.

Advantages provided by the image processing device 1 according to thepresent embodiment will now be described. As described above, the imageprocessing device 1 can generate an image plane-synthesized for digitalbroadcasting (FIG. 8) and a stereoscopic image (FIGS. 12 to 14) bychanging the input data (first image data 21, second image data 22, maskinformation 23, etc.) and setup information (data selection information303, display position information 305, etc.) as needed.

The mixer unit 101 and other processing units are capable of generatinga stereoscopic image and achieving a plane synthesis for digitalbroadcasting. This eliminates the necessity of adding a dedicatedprocessing unit for stereoscopic images and a dedicated processing unitfor plane synthesis to the image processing device 1. This makes itpossible to simplify and downsize the device and reduce its cost and itsmaintenance cost.

Further, the image processing device 1 can support side-by-side inputimage data and side-by-side output image data as described earlier byallowing the scale processing units 30, 31 to rescale the first imagedata 21 and the second image data 22.

The prevent invention is not limited to the foregoing embodiments, butextends to various modifications that nevertheless fall within the scopeof the appended claims.

What is claimed is:
 1. An image processing device comprising: a firstmixer unit; a first scale processing unit which processes first imagedata, generates a first output image and a first enable signal, andsupplies the first output image and the first enable signal to the firstmixer unit; a second mixer unit which receives an output of the firstmixer unit; a second scale processing unit which processes second imagedata, generates a second output image and a second enable signal, andsupplies the second output image to the second mixer unit; a third scaleprocessing unit which processes mask information, and generates a thirdoutput image and a third enable signal; an enable signal processing unitwhich receives the second enable signal, the third output image and thethird enable signal, and generates a fourth enable signal and suppliesthe fourth enable signal to the second mixer unit; a fourth scaleprocessing unit which processes text/image data, and generates a fourthoutput image and a fifth enable signal; and a third mixer unit whichreceives the output of the second mixer unit, the fourth output imageand the fifth enable signal, and selects one of the output of the secondmixer unit and the fourth output image to generate a composite image. 2.The image processing device of claim 1, wherein the first scaleprocessing unit processes the first image data by cutting out arectangular portion from the first image data, and enlarging or reducingthe rectangular portion to a predetermined size, the first output imagecomprising the enlarged or reduced rectangular portion of the firstimage data.
 3. The image processing device of claim 1, wherein thesecond scale processing unit processes the second image data by cuttingout a rectangular portion from the second image data, and enlarging orreducing the rectangular portion to a predetermined size, the secondoutput image comprising the enlarged or reduced rectangular portion ofthe second image data.
 4. The image processing device of claim 1,wherein the third scale processing unit processes the mask informationby cutting out a rectangular portion from the mask information, andenlarging or reducing the rectangular portion to a predetermined size,the third output image comprising the enlarged or reduced rectangularportion of the mask information.
 5. The image processing device of claim1, wherein the fourth scale processing unit processes the text/imagedata by cutting out a rectangular portion from the text/image data, andenlarging or reducing the rectangular portion to a predetermined size,the fourth output image comprising the enlarged or reduced rectangularportion of the text/image data.
 6. The image processing device of claim1, wherein the composite image is obtained by placing the first andsecond image data in the background and placing the text/image data inthe foreground.
 7. The image processing device of claim 1, wherein thefirst scale processing unit comprises: a cutting unit which reads dataselection information which indicates pixels which are to be cut outfrom the first image data, and cuts out an image defined by the dataselection information from the first image data; and anenlarging/reducing unit which: receives the cut-out image from thecutting unit, reads display position information indicating a displayregion within the first output image in which the cut-out image is to bedisplayed, and enlarges or reduces the cut-out image in accordance withthe display position information to generate the first output image; andgenerates the first enable signal which indicates an effective region ofthe first output image.
 8. The image processing device of claim 7,wherein a bit value “0” or a bit value “1” is set for each pixel in thedata selection information.
 9. The image processing device of claim 7,wherein the first enable signal has the same bit pattern as the displayposition information.
 10. The image processing device of claim 1,wherein a bit value of each pixel in the first enable signal is used forpixel value selection in the first, second and third mixer units. 11.The image processing device of claim 1, wherein the enable signalprocessing unit comprises: a NAND gate having the third output image andthe third enable signal as inputs; and an AND gate having an output ofthe NAND gate and the second enable signal as inputs, and supplying alogical sum of the output of the NAND gate and the second enable signalas the fourth enable signal to the second mixer unit.
 12. The imageprocessing device of claim 11, wherein if a bit value of the thirdenable signal is “0”, then a bit value of the second enable signal isoutput as the fourth enable signal without regard to the bit value ofthe third output image, and wherein if a bit value of the third enablesignal is “1”, then a bit value of the third output image is output asthe fourth enable signal.
 13. The image processing device of claim 1,wherein background information is input to the first mixer unit, thebackground information comprising image data for which a predeterminedplain color is designated.
 14. The image processing device of claim 13,wherein if a bit value of the first enable signal is “0”, then the firstmixer unit outputs a pixel value at relevant coordinates of thebackground information as the pixel value at the relevant coordinates,and wherein if a bit value of the first enable signal is “1”, then thefirst mixer unit outputs a pixel value at relevant coordinates of thefirst output image as the pixel value at the relevant coordinates. 15.The image processing device of claim 1, wherein in accordance with thefourth enable signal output from the enable signal processing unit, thesecond mixer unit selects and outputs either a pixel value output fromthe first mixer unit or a pixel value output from the second scaleprocessing unit.
 16. The image processing device of claim 15, wherein ifthe fourth enable signal is “0”, then the second mixer unit selects apixel value output from the first mixer unit and outputs the selectedpixel value to the third mixer unit, and wherein if the fourth enablesignal is “1”, then the second mixer unit selects a pixel value outputfrom the second scale processing unit and outputs the selected pixelvalue to the third mixer unit.
 17. The image processing device of claim1, wherein if a bit value of the fifth enable signal is “0”, then thethird mixer unit selects a pixel value output from the second mixer unitand sets the selected pixel value as the pixel value of a relevant pixelof the composite image, and wherein if a bit value of the fifth enablesignal is “1”, then the third mixer unit selects a pixel value outputfrom the fourth scale processing unit and sets the selected pixel valueas the pixel value of the relevant pixel of the composite image.